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ABSTRACT 

A  program  using  digital  correlation  techniques  has  been 
implemented  on  the  Digital  Image  Analysis  Laboratory  (DIAL) 
to  automatically  extract  elevation  type  data  from  stereo¬ 
scopic  pairs  of  digital  images.  The  DIAL  permits  the 
operator  to  view  on  a  display  unit  the  stereoscopic  model 
and  the  computed  match-point  coordinates  in  3-D  using  ana- 
glyphic  techniques.  The  match  points  are  presented  as  3-D 
profiles  superimposed  over  the  3-D  terrain  model  and  conse¬ 
quently  the  operator  can  observe  the  compilation  process, 
detect  errors  as  soon  as  they  occur,  and  interact  with  the 
process  to  prevent  the  accumulation  of  error  or  to  tune 
the  parameters  as  conditions  warrant.  The  program  is  de¬ 
scribed  in  terms  of  the  hardware  used,  the  algorithm,  the 
operating  procedures,  and  an  example  of  a  compiled  area. 

INTRODUCTION 

Over  the  past  ten-plus  years  the  Computer  Sciences  Labora¬ 
tory  (CSL)  of  the  Engineer  Topographic  Laboratories  (ETL) 
has  sponsored  in-house  and  private  industry  efforts  to 
develop  digital  techniques  that  can  be  applied  to  extract 
elevation  data  automatically  from  a  stereoscopic  pair  of 
photographs.  One  of  the  first  efforts  was  performed  for 
ETL  by  Keuffel  and  Esser  Company  (K&E)  (Rosenberg,  1971,74) 
and  was  aimed  towards  the  development  and  testing  of  an 
algorithm  for  digital  correlation  using  the  "infiltration 
method"  -  a  complex  process  that  follows  the  path  of 
easiest  correlation  rather  than  a  defined  path  in  local 
space.  Much  of  the  philosophy  developed  by  the  K&E  effort 
was  used  later  in  a  comprehensive,  five-phase  study  by 
Control  Data  Corporation  (CDC)  (Panton,  1978)  aimed  at 
algorithms  development,  implementation  of  the  algorithms 
on  special  purpose,  high-speed  "Flexible  Processors" 
actual  testing  on  stereoscopic  imagery  and  development  of 
preliminary  specifications  for  a  digital  stereoscopic  image 
processing  system. 

Prior  to  and  overlapping  the  CDC  effort,  in-depth  studies 
were  performed  in-house  (Crombie,  1974,75,76)  to  isolate 
the  cause-effect  relationships  between  digital  correlation 
results  and  changes  to  the  various  combinations  of  parame¬ 
ters  that  can  be  used  in  a  digital  stereoscopic  image 
matching  process.  All  of  the  studies  involved  software 
that  was  run  in  a  batch  mode  and,  consequently,  the  results 
had  to  be  evaluated  using  laborious,  time-consuming  statis¬ 
tical  analyses  of  large  amounts  of  data. 


In  order  to  more  readily  examine,  evaluate  and  edit  the 
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data  from  the  in-house  studies,  software  was  generated  for 
the  Digital  Image  Analysis  Laboratory  (DIAL)  which  was  be¬ 
ing  developed  at  ETL  in  the  general  time  frame  that  the 
CDC  and  in-house  studies  were  conducted.  With  the  advent 
of  the  DIAL,  which  is  to  be  described  later,  a  means  was 
available  for  image  display  and  manipulation  along  with 
the  opportunity  to  interact  with  the  software  controlling 
the  computational  process.  This  modest  effort  aimed  at  the 
visual  examination  of  batch  mode  results  soon  evolved  into 
a  complete  DIAL  compilation  program  which,  hereafter,  will 
be  referred  to  as  the  Digital  Interactive  Mapping  Program 
(DIMP) . 

The  DIMP  was  generated  to  serve  the  following  purposes: 

Minimize  the  time  required  to  perform  digital  corre¬ 
lation  tests  and  to  evaluate  results. 

.  Perform  studies  for  algorithm  development. 

.  Test  means  for  visual  display  of  results  and  for  data 
editing. 

.  Demonstrate  an  automatic,  but  interactive,  means  for 
the  compilation  of  elevation  data  using  digital  image  pro¬ 
cessing  techniques. 

In  the  implementation  of  the  DIMP,  several  of  the  features 
suggested  by  the  CDC  studies  were  incorporated;  especially 
in  regard  to  the  stereoscopic  presentation  of  the  digital 
images,  stereoscopic  presentation  of  the  correlation  re¬ 
sults  and  data  editing  techniques.  On  the  other  hand,  the 
results  obtained  in  the  in-house  efforts  were  relied  on 
heavily  for  the  implementation  of  an  algorithm  and  in  the 
selection  of  options  and  parameters,  such  as  window  sizes, 
point  spacing,  quality  checks,  etc.  The  DIMP  is  described 
in  the  following  sections  in  terms  of  the  equipment  used, 
the  algorithm,  operating  procedures  and  examples  of  results 
obtained  with  the  program. 

THE  DIAL 

The  Digital  Image  Analysis  Laboratory  (DIAL)  is  a  large- 
scale,  interactive  system  designed  for  research  and  devel- 
^nent  activities  in  digital  image  processing.  It  is  part 
c i  the  ETL  computer  facility  which  consists  primarily  of  a 
host  computer  subsystem,  a  STARAN  subsystem  and  an  inter¬ 
active  subsystem.  The  Control  Data  Corporation  CDC- 6400 
computer  is  used  in  a  time-sharing  mode  with  other  ETL 
users  as  the  host  computer  for  the  DIAL  system  and  thereby 
performs  practically  all  of  the  processing  and  control 
functions.  The  Goodyear  Aerospace  STARAN  computer  is  an 
associative  array  processor  that  is  used,  because  of  its 
high-speed  and  parallel-processing  capabilities,  to  perform 
computationally  bound  tasks.  It  is  not  used  with  the  DIMP 
program  and,  therefore,  will  not  be  discussed  further. 

The  interactive  subsystem  of  the  DIAL  is  that  part  which 
makes  the  DIMP  possible  since  it  permits  the  operator  to 
display  and  view  the  digital  photographic  images  and  the 
correlation  results  and,  when  deemed  necessary,  to  inter¬ 
rupt  the  processor  to  change  program  parameters  and  vari¬ 
ables. 

The  interactive  subsystem  consists  mainly  of  two  work 
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stations,  one  of  which  is  shown  in  Figure  1.  Both  are  con- 


Figure  1.  DIAL  Work  Station. 


trolled  by  a  DEC  PDP  11/50  computer  that  is  interfaced  to 
the  CDC  6400.  The  host  computer  does  practically  all  of 
the  processing  but  some  simple  local  operations  can  be  done 
concurrently  on  the  PDP  11/50.  Each  of  the  two  work  sta¬ 
tions  consists  of  two  COMTAL  8300  display  monitors,  a 
TEKTRONIX  alphanumeric  terminal,  a  XY-tablet  and  one  track¬ 
ball  unit  per  monitor.  One  TEKTRONIX  hardcopy  device  is 
used  to  copy  information  from  both  terminals. 

Each  display  unit  features  refresh  storage  of  three  digital 
image  memories  characterized  by  an  array  of  512  x  512 
elements  (or  pixels)  whose  grey  shades  can  vary  in  magni¬ 
tude  from  0  to  255  (8-bits) .  Each  monitor  also  has  three 
1-bit,  512- x  512-pixel  overlay  memories  and  one  1-bit, 
512-x  512-pixel  fast  graphics  memory.  The  graphic  over¬ 
lays  are  used  to  superimpose  graphic  images  over  the  8 -bit 
images  in  color;  and  the  fast  graphics  plane  is  used  to  plot 
graphic  symbols  such  as  dots,  lines,  circles,  etc.  The 
monitors  can  be  operated  in  a  black  and  white  or  true-color 
mode  at  the  discretion  of  the  operator.  In  the  DIMP  the 
true-color  mode  is  used  to  display  the  left  image  of  a 
stereoscopic  pair  in  both  blue  and  green  to  give  cyan.  The 
right  image  is  displayed  in  red.  By  using  the  anaglyphic 
glasses,  the  operator  can  view  the  superimposed  red  and 
cyan  images  in  three-dimensions  (3-D) .  A  3-D  profile  can 
be  viewed  by  plotting  one  line  in  cyan  and  the  other  in  red 
on  the  graphic  overlays  and,  likewise,  a  "floating"  dot  can 
be  obtained  by  using  a  red  cursor  and  an  equivalent  size 
dot  plotted  in  cyan  on  the  fast  graphics  plane.  The  pro¬ 
file  and  dot  applications  are  described  later  in  more 
detail. 


THE  IMAGES 

Practically  all  experiments  run  with  the  DIMP  involved  a 
stereoscopic  pair  of  images  called  REC01  and  REC02.  These 
are  2-x  2- inch  subsets  taken  from  9-x  9-inch  photographs 
of  the  Phoenix,  Arizona  area.  The  photographs  were  aligned 


and  digitized  on  a  microdensitometer  comparator  with  the 
direction  of  flight  parallel  to  the  scanning  axis  of  the 
comparator.  There  is,  therefore,  practically  no  skew  be¬ 
tween  images  and  no  Y-parallax  since  the  original  photo¬ 
graphs  are  at  the  same  scale  and  essentially  vertical.  The 
images  were  digitized  at  a  pixel  and  line  spacing  of  24  um 
(micrometer)  using  a  spot  size  of  34.5  um.  A  total  of  2048 
lines  were  recorded  with  a  10-bit  representation  of  each 
grey-shade  value.  The  digitized  data  were  recorded  on  tape 
and,  as  required  by  the  DIAL,  transferred  to  a  disk  pack. 

The  original  photographs  were  taken  from  an  altitude  of 
approximately  23,500  feet  with  a  6-inch  focal  length  camera. 
The  scales  are  therefore  about  1:47,000  and  each  pixel  cov¬ 
ers  about  3.8  feet  on  the  ground.  Wien  displayed  on  the 
10-inch  screen  of  the  monitor,  the  viewing  scale  of  the 
digitized  imagery  is  approximately  five  times  larger  than 
the  original  or  1:9400.  When  a  subset  of  512  x  512  is 
taken  from  the  2048  x  2048  image  it  is  displayed  at  a  view¬ 
ing  scale  of  1:2350. 


THE  ALGORITHM 


General . 

Given  the  interior  and  exterior  orientation  data  and  the 
calibration  data  for  a  stereoscopic  pair  of  photographs,  it 
is  necessary  only  to  obtain  the  corresponding  X  and  Y  pho¬ 
tocoordinates  of  an  image  on  the  stereoscopic  pair  in  order 
to  compute  its'  ground  position.  The  digital  correlation 
algorithm  described  herein  is  used  to  obtain  the  necessary 
corresponding  photocoordinates. 

In  this  scheme  a  window  of  grey- shade  values  centered 
around  the  point  in  question  on  the  left  photograph  is  com¬ 
pared  at  every  possible  position  within  a  specified  search 
area  to  the  grey  shades  on  the  right  photograph.  The  posi¬ 
tion  within  the  search  area  where  the  maximum  correlation 
occurs  will  define,  to  a  pixel  precision,  the  location  on 
the  right  photograph  of  the  corresponding  image.  A  curve 
is  fitted  in  both  the  X  and  Y  directions  to  the  maximum 
correlation  value  and  ones  on  either  side  and  then  analyzed 
to  determine  to  a  fraction  of  a  pixel  where  the  peak  occurs. 
The  specified  pixel  coordinates  of  the  image  on  the  left 
photograph  and  the  computed  pixel  position  on  the  right 
photograph  can  be  transformed  off-line  to  actual  photoco¬ 
ordinates  and  subsequently  used  to  compute  X-jY-and  Z- 
ground  coordinates. 

Some  of  the  individual  considerations  that  must  be  made 
to  implement  the  continuous  and  automatic  compilation  of 
match  (or  corresponding)  points  are  discussed  in  the  fol¬ 
lowing  sections. 

Window  and  Search  Area  Sizes 

As  a  matter  of  definition,  the  term  "window"  refers  to  that 
rectangular  array  of  grey  shades  on  the  left  photograph 
that  is  symmetrical  relative  to  the  pixel  position  for 
which  a  match  point  is  required.  The  "search"  area  is  a 
rectangular  array  of  grey  shades  on  the  right  photograph 
symmetrically  centered  about  that  pixel  position  where  the 
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match  point  is  estimated  to  occur.  The  estimated  point  in 
the  search  area  is  refined  by  the  correlation  process  to 
give  a  computed  match  point . 

The  window  size  used  in  the  DIMP  is  generally  15  by  15 
pixels  although  the  operator  has  the  option  of  selecting 
any  practical  size  and  shape  of  window.  The  search  area 
is  generally  two  lines  and  six  pixels  (17  by  21)  larger 
than  the  window  but  any  size  can  be  used.  These  sizes  are 
large  enough  to  give  good  results  yet  small  enough  to  allow 
reasonable  computational  speed.  They  are  initially  select¬ 
ed  by  the  operator  but  they  may  be  altered  under  program 
control  when  the  program  logic  detects  conditions  which 
call  for  larger  or  smaller  array  sizes. 

As  illustrated  in  Figure  2,  a  15  x  15  window  passed  over  a 
17  x  21  search  area  results  in  three  rows  of  correlation 
values  in  the  Y-parallax  direction  and  seven  columns  of 
values  in  the  X-parallax  direction.  This  allows  for  plus 
or  minus  one  pixel  of  Y  parallax  removal  and  plus  or  minus 
three  pixels  correction  to  the  estimated  X  position  of  the 
match  point. 


15x15  WINDOW 


m 

1 

U 

£ 

£ 

m 

I 

u 

iffe 

ft 

£ 

a 

£ 

£ 

ft 

« 

B 

a 

B 

— 

— 

a 

T 

r 

□ 

□ 

r 

J 

:: 

r 

— 

FI 

1 

DCT  I 

2 

B 

B 

■ 

m 

m 

«  i 

Mil 

n 

1_ 

j 

m 

a 

a 

3 

II 

IB 

MAT 

HXEC m 

m 

ct 

ii 

IB 

BB 

F/BB 

m 

m 

K 

■ 

■ 

a 

t] 

IB 

BB 

fiBI 

a 

m 

B 

B 

Yt 

1 

□ 

one? 

a 

□□B 

:■ 

r 

■ 

a 

a 

□□ 

□ 

□□□□ 

BB 

B 

- 17 

OW 

■ 

a 

□ 

n 

!□□□□□□ 

mm 

B 

a 

1  / 

■ 

j 

LI 

■HI 

m 

>M 

. 

. 

a 

r 

: 

vA 

RE 

Ui 

— w— 

W  ^i!VTW  'M-.’iiTin.* 

ft 

m 

a 

a 

a 

a 

LA 

u 

n  r 

LJ 

m 

Mil 

UI 

m 

O' 

4 

j 

>» 

J 

r 

r 

r 

4 

4 

j 

* 

a 

a 

n 

a 

a 

a 

m 

a 

a 

a 

m 

m 

ft 

•M 

B 

B 

a 

a 

zz 

a 

s 

u 

• 

& 

* 

m 

& 

■i 

m 

& 

m 

m 

B 

B 

B 

II 

_ 

_ 

_ 

_ 

L 

_ 

_ 

_ 

_ 

_ 

_ 

_ 

_ 

I 

x21  SEARCH  AREA 


Figure  2.  Relationship  Between  the  Window  and  Search  Area. 
Point  Spacing. 

The  compilation  process  proceeds  from  left  to  right  and 
from  top  to  bottom  at  equal,  operator-selected  increments 
(NCRE)  on  the  left  photograph.  The  spacing  on  the  right 
photograph  will  be  varied  because  of  image  displacement 
due  primarily  to  relief  displacements.  Since  match  points 
are  determined  in  a  regular  array  in  image  space,  a  post¬ 
processing  scheme  must  be  used  to  transform  any  subsequent 
elevation  data  into  an  even  array  in  ground  space. 

Any  point  spacing,  NCRE,  can  be  used  in  the  DIMP  but,  as 
described  in  the  following  section,  options  such  as  window 
shaping  depend  on  the  point  spacing.  If  the  point  spacing 
is  too  large  in  comparison  with  the  window  size,  window 
shaping  cannot  be  performed  accurately.  A  five -or  seven- 
pixel  spacing  has  been  used  most  frequently  in  the  DIMP 
because  they  are  consistent  with  other  available  options 
and  provide  acceptable  speeds  of  operation. 
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Window  Shaping 

Since  Stereopnotos  are  taken  from  different  perspectives, 
a  length  of  sloping  terrain  Imaged  on  the  left  photograph 
will  be  different  in  size  and  shape  than  its  image  on  the 
right  photograph.  In  order  to  make  the  two  images  more 
alike,  one  or  the  other  must  be  reshaped.  Since  match- 
point  computations  are  performed  in  image  space,  only  one 
image  needs  to  be  shaped  relative  to  the  other.  Ground 
space  computations  would  require  that  both  images  be  shaped, 
resulting  in  unnecessary  computation  time. 

The  smaller  window  of  the  left  photo  is  shaped  to  match  the 
search  area  on  the  right  photograph  using  the  inverse 
equations  (1): 

Y  =  YT  (1) 

X  -  (XT  -  B*YT) /A 

where  XT,  YT  -  New  pixel  coordinates 

X,  Y  -  Original  pixel  coordinates 

A,  B  -  Transformation  parameters 

The  transformation  parameters  A  and  B  are  computed  on  the 
basis  of  previous  match-point  results.  In  the  general  case, 
the  previous  four  columns  to  the  left  of  the  "next"  point 
and  the  previous  four  rows  are  used  in  the  computation. 
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Figure  3.  Geometric  Basis  for  Window 
Shaping  and  Prediction  of  "Next"  Point . 

Referring  to  Figure  3,  the  DX  values  represent  the  dis- 
..ances  between  match  points  on  the  right  photograph  in  the 
direction  of  X  parallax.  A  weighted  average  of  the  DX 
values  are  used  to  compute  the  scale,  A,  relative  to  the 
incremental  point  spacing,  NCRE,  on  the  left  photograph: 


DX12  *  (DX1* 1+DX2* 2+ - DX10*10+DX11*11) /66 

A  *  DX12/NCRE  (2A) 

Likewise,  the  skew  between  images,  B,  is  computed  as 
follows: 

DY12  -  (DY1*1+DY2*2H - DY10*10+DY11*11) / 66 

B  -  DY12/NCRE  (2B) 


The  weighted  computations  are  used  to  give  the  previous, 
closer  match-point  results  more  weight  but  yet  include 
further  points  in  order  to  minimize  the  effect  of  errors 
in  the  immediate  vicinity  of  the  "next"  point.  The  A  and 
B  computations  are  based  on  only  four  columns  and  four 
rows  of  previous  results  in  order  to  minimize  computer 
storage  requirements.  Modified  versions  of  equations  (2A) 
and  (2B)  are  used  when  less  than  four  rows  and  four  columns 
have  been  compiled  but  the  approach  is  generally  the  same. 

Match-Point  Prediction. 

The  success  of  the  DIM?  depends  largely  on  how  accurately 
the  match-point  location  on  the  right  photograph  can  be 
predicted.  The  correlation  process  is,  after  all,  a  local 
refinement  to  an  estimated  location  and  not  a  global  search 
operation.  The  Y  coordinate  of  the  first  point  in  each  row 
(1,1)  is  determined  simply  by  incrementing  the  Y  coordinate 
of  the  match  point  above  (1-1,1)  by  the  point  spacing,  NCRE. 
This  assumes  the  two  digital  images  are  at  the  same  scale. 
The  Y  coordinate  of  each  point  after  the  first  is  consid¬ 
ered  to  be  the  same  as  that  of  the  previous  point  (I,J-1). 

The  X  coordinate  of  the  "next"  point  is  determined  using 
the  weighted  distance,  DX12,  as  described  in  the  previous 
section.  As  shown  in  Figure  3,  the  DX12  value  is  added  to 
the  X  value  of  the  previous  point  in  the  same  row  (I,J-1) 
to  give  one  estimate  for  the  next  point.  A  second  estimate 
for  the  next  point  is  obtained  by  projecting  a  straight 
line  through  the  three  previous  points  (1-3 , J ; 1-2 , J ; 1-1 , J) 
in  the  same  column  of  match  points.  The  two  estimates  are 
averaged  and  the  result  is  used  for  the  X  coordinate  of 
the  next  point  (I,J). 

At  the  beginning  of  the  correlation  process  modifications 
to  the  above  techniques  are  used  to  compute  the  location 
of  the  next  point  until  four  rows  and  four  columns  of 
match  points  have  been  determined.  For  the  first  row  of 
points,  and  the  first  point  of  each  row,  the  process  uses 
the  X  and  Y  coordinates  determined  by  the  operator  in  the 
"starting  profiles"  procedure  described  in  a  later  section. 
For  rows  2  and  3,  the  X  values  are  computed  using  only  the 
first  estimate  for  the  point  based  on  the  DX  values  avail¬ 
able  thus  far. 

Correlation  Coefficients. 

The  operator  has  the  option  of  using  the  linear  correlation 
coefficient  (RXY) ,  the  covariance  correlation  coefficient 
(SXY)  or  the  absolute  difference  coefficient  (DXY)  to 
determine  the  degree  of  correlation  between  the  window  and 
the  search  area.  The  three  coefficients  are  computed  as 
follows : 


RSY  =  cxy  =  E(X-X)  (Y-Y) 

axay  , - = -  « - — — 

/e(x-x)2  '/e(y-y)2 

SXY  -  oxy  -  yj  E(X-X)(Y-Y) 


(X-X)-(Y-Y) 

where  X,Y  . the  grey-shade  values  for  the  window  and 

search  area. 

X,Y . the  average  X  and  Y. 

N . number  of  elements  in  the  window. 

Because  the  RXY  value  is  more  consistent  and  predictable, 
it  was  used  almost  exclusively  in  the  DIMP  in  spite  of  the 
fact  that  it  is  slower  and  more  costly  to  compute. 

Quality  Tests. 

The  quality  of  correlation  is  judged  good  or  bad  based  on 
the  magnitude  of  the  computed  correlation  coefficient  and 
also  on  the  magnitude  of  correction  the  correlation  process 
makes  to  the  predicted  location  of  the  match  point.  The 
terms  "good"  and  "bad"  are  used  here  to  indicate  conform¬ 
ance  to  statistical  tests  and  not  as  an  absolute  measure  of 
quality.  When  the  correlation  process  succeeds  in  passing 
the  quality  test,  a  value  of  zero  is  assigned  to  the  qual¬ 
ity  of  that  point.  If  it  is  not  successful,  a  value  of  1 
is  added  to  the  quality  value  of  the  previous  point  in  the 
same  column  of  match  points.  If,  for  example,  the  correla¬ 
tion  process  has  failed  five  times  along  a  vertical  column 
of  match  points,  the  quality  of  the  fifth  point  is  5.  The 
operator  can  set  a  limit  on  how  many  successive  times  the 
correlation  process  can  fail.  When  the  limit  is  reached, 
the  automatic  process  halts  and  signals  the  operator  to 
intervene,  if  necessary,  in  order  to  get  the  process  back 
on  track.  The  operator  can  also  intervene  at  will  in  order 
to  circumvent  the  accumulation  of  bad  points  in  obviously 
poor  correlation  areas.  Just  how  the  operator  gets  the 
correlation  process  back  on  track  is  described  later  in  the 
section  on  operating  procedures. 

Iterative  Techniques. 

The  operator  can  specify  the  maximum  number  of  iterations 
that  will  be  allowed  in  order  to  obtain  acceptable  correla¬ 
tion  results  on  each  point.  The  logic  used  in  the  itera¬ 
tive  technique  is  given  in  Figure  4.  The  extra  iterations 


Figure  4.  Iterative  Software  Logic. 
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are  allowed  when  successful  correlation  is  not  obtained  on 
the  first  try.  Depending  on  the  quality  of  the  previous 
point,  the  logic  allows  the  window  and/or  search  areas  to 
increase  or  decrease  in  size.  The  maximum  allowable  cor¬ 
rections  to  the  computed  match-point  position  may  also  be 
increased.  The  multiple  iteration  logic  changes  the  corre¬ 
lation  philosophy  from  one  of  refinement  to  a  predicted 
position  to  that  of  a  limited  search  over  a  small  area. 

Match-Point  Adjustment. 

After  the  DIMP  completes  the  correlation  process  of  all 
points  in  a  given  row,  the  quality  of  each  point  in  the  row 
is  checked  to  determine  if  any  were  judged  to  be  bad.  If 
bad  points  are  encountered,  a  straight  line  interpolation 
is  done  between  the  good  points  on  either  side  of  the  bad 
ones  to  compute  an  adjusted  match  point  coordinate  for  the 
bad  points.  This  procedure  is  illustrated  in  Figure  5. 
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Figure  5.  Match-Point  Adjustment  Scheme. 

The  interpolation  scheme  takes  advantage  of  the  fact  that, 
for  the  most  part,  the  terrain  is  continuous  and  uniform 
over  short  distances.  The  interpolated  values  are  consid¬ 
ered  to  be  better  than  the  "bad"  points;  but,  for  the  pur¬ 
pose  of  the  iterative  logic  described  in  the  previous  sec¬ 
tion,  the  points  still  are  judged  to  be  bad. 

OPERATING  PROCEDURES 

Image  Display. 

The  initial  step  in  the  operation  of  the  DIMP  is  to  enter 
through  the  keyboard  of  the  TEKTRONIX  alphanumeric  terminal 
the  names  of  the  two  photographs  that  constitute  the 
stereoscopic  pair.  The  computer  reads  the  digital  data 
from  a  disk  pack  and  makes  the  images  available  for  display 
on  one  of  the  display  units.  As  is  the  case  of  REC01  and 
REC02,  images  larger  than  512  x  512  pixels  are  reduced  in 
scale  so  that  the  full  image  will  fit  on  the  screen.  Sub¬ 
sets  are  selected  from  the  reduced  images  and  displayed  on 
the  second  monitor  at  full  resolution.  Using  the  color 
capability  of  the  monitor,  the  left  subimage  is  displayed 
in  cyan  and  the  right  subset  in  red.  The  operator  then 
uses  anaglyphic  glasses  to  view  the  subset  imagery  as  a  3-D 
stereoscopic  model. 
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Floating  Dot. 

The  cursor  and  fast  graphic  capabilities  of  the  monitors 
make  it  possible  to  superimpose  a  "floating  dot"  in  the  3-D 
model.  The  floating  dot  is  constructed  in  the  DIMP  by  de¬ 
fining  the  cursor  as  a  3  x  3-pixel  array  whose  color  is 
red.  In  conjunction  with  the  cursor,  a  3  x  3-pixel  array 
is  plotted  in  cyan  on  the  fast  graphics  plane  of  the  moni¬ 
tor.  The  trackball  associated  with  one  monitor  is  used  to 
move  the  cursor  and  the  fast  graphics  dot  equally  through¬ 
out  the  stereoscopic  model.  The  second  trackball  is  used 
to  move  the  fast  graphics  dot  in  the  X-  and  Y-parallax 
directions  relative  to  the  red  cursor.  With  the  anaglyphic 
glasses  on,  the  operator  sees  a  white  floating  dot  which  can 
be  moved  throughout  the  model  area  and,  by  adjusting  the 
parallax  between  the  red  and  cyan  dots,  can  be  "raised"  or 
f'lowered"  in  elevation  to  appear  to  rest  on  the  3-D  surface. 

Parameter  Selection . 

Once  the  full-resolution  subset  of  imagery  to  be  compiled 
has  been  selected  and  displayed,  the  operator  is  prompted 
by  the  computer  to  enter  the  values  for  the  parameters  dis¬ 
cussed  in  the  previous  sections  on  "The  Algorithm."  During 
execution  of  the  program,  the  operator  can  stop  the  program 
and  change  these  parameters  to  better  suit  the  terrain  con¬ 
ditions  but  normally  this  is  not  necessary. 

Starting  Profiles. 

The  operator  has  the  option  of  compiling  as  much  or  as 
little  of  the  512-x  512-pixel  area  as  desired.  Large 
areas  would  normally  be  selected  but  in  special  cases,  such 
as  recompilation  of  bad  areas  or  the  study  of  select  terrain 
features,  small  areas  are  desired.  In  any  case  the  opera¬ 
tor  selects  the  beginning  point  at  the  top  left  corner  of 
the  desired  area  by  placing  the  floating  dot  precisely  on 
the  3-D  surface  at  that  point  and  recording  its'  coordinates 
on  the  left  and  right  images  by  pushing  a  button  on  the 
trackball  unit.  The  operator  then  proceeds,  from  left  to 
right,  to  plot  a  profile  of  the  major  breaks  in  the  terrain 
surface.  The  procedure  is  repeated  from  top  to  bottom 
starting  at  the  previously  defined  beginning  point  on  the 
left  edge  of  the  compilation  area.  The  DIMP  then  interpo¬ 
lates  between  the  gross  profile  points  to  compute  "given" 
estimated  match-point  coordinates  at  every  increment  along 
the  profile  that  corresponds  to  the  previously  selected 
point  spacing  between  correlation  points.  The  interpolated 
match-point  coordinates  are  used  as  the  "predicted  match 
points"  described  previously  and  also  are  used  for  window 
spacing  of  the  first  row  in  the  correlation  process. 

Automatic  Compilation. 

The  correlation  process  is  automatic  until  the  operator 
intervenes  or  the  program  stops  due  to  loss  of  correlation. 
Starting  at  the  first  row,  the  correlation  process  proceeds 
left  to  right  across  the  area  designated  by  the  operator 
when  the  starting  profiles  were  drawn.  At  the  conclusion 
of  each  match-point  determination,  the  floating  dot  is 
positioned  at  that  match  point  until  the  next  match  point 
is  determined.  The  operator  therefore  observes  the  dots 
moving  from  left  to  right  and  "up"  and  "down"  as  it  follows 
the  shape  of  the  terrain  surface. 
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At  the  completion  of  each  row  of  correlation  points,  lines 
are  drawn  columnwise  between  the  last  two  completed  rows  of 
match  points.  On  the  left  image  the  lines  are  vertical  and 
evenly  spaced  since  the  points  are  predesignated  when  the 
operator  selects  the  incremental  spacing  between  correla¬ 
tion  points.  These  lines  are  drawn  in  cyan  using  the  fast 
graphics  plane  of  the  COMTAL  display  and  are  superimposed 
over  the  cyan  terrain  image.  Red  lines  are  drawn  column¬ 
wise  on  a  graphic  overlay  between  the  last  two  completed 
rows  of  computed  match  points  and  are  superimposed  over  the 
red  terrain  image.  These  lines  are  not  straight  nor  evenly 
spaced  because  the  match-point  coordinates  reflect  the  X 
parallax  between  images  due  mainly  to  changes  in  elevation. 
Since  the  cyan  and  red  lines  pass  through  corresponding 
images,  the  operator  perceives  3-D  profile  lines  that 
follow  the  terrain  surface.  If  the  correlation  results  are 
in  error,  the  lines  will  appear  to  "float"  above  or  below 
the  terrain  model  surface.  The  operator  uses  this  visual 
display  of  the  correlation  results  to  determine  if  and  when 
manual  intervention  is  needed  in  order  to  prevent  the  prop¬ 
agation  of  errors  in  the  compilation  process. 

The  lines  are  plotted  at  the  completion  of  each  row  of 
correlations  and,  therefore,  the  lengths  of  the  lines  grow 
progressively  with  time.  This  type  of  display  permits  the 
operator  to  study  casually  the  previous  results  obtained 
in  the  stereoscopic  model  and  also  to  anticipate  well  ahead 
just  what  problems  may  be  encountered  due  to  upcoming 
changes  in  the  terrain  surface.  If  no  operator  interven¬ 
tion  is  necessary,  the  automatic  mode  continues  until  the 
area  selected  by  the  operator  is  completed.  Before  proceed¬ 
ing  to  compile  a  new  sub image,  the  operator  will  normally 
study  the  visual  results  on  the  display  and  decide  whether 
or  not  there  are  any  areas  of  the  stereoscopic  model  that 
were  not  compiled  satisfactorily.  In  the  presence  of  bad 
results,  the  operator  can  correct  the  data  before  continu¬ 
ing  with  new  areas  or  elect  to  correct  the  data  at  a  later 
date  and  possibly  off-line  from  the  DIMP. 

EXAMPLE  OF  RESULTS 

One  of  the  more  desirable  aspects  of  the  DIMP  is  that  the 
correlation  results  are  displayed  visually  and  therefore 
afford  the  operator  the  opportunity  to  rapidly  judge  the 
quality  without  dependence  on  laborious  statistical  anal¬ 
yses.  The  operator  views  the  results  with  anaglyphic 
glasses  but,  for  documentation  purposes,  the  cyan  and  red 
lines  and  images  can  be  displayed  in  black  and  white,  pho¬ 
tographed,  and  made  into  stereograms  suitable  for  publica¬ 
tion  purposes.  One  such  stereogram  is  shown  in  Figure  6 
to  illustrate  the  manner  in  which  match  point  results  are 
displayed. 

The  results  shown  in  Figure  6  were  compiled  using  a  15  x  15 
window,  a  17  x  21  search  area,  a  two-pixel  correction  to 
the  predicted  match  point  and  eight  consecutive  losses 
before  the  automatic  process  would  stop.  Also,  four  itera¬ 
tions  per  point  were  allowed,  window  shaping  was  performed 
and  the  linear  correlation  coefficient  (RXY)  was  used. 

Match  points  were  computed  at  increments  of  every  seven 
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Figure  6.  Stereogram  of  Compilation  "Standard". 


pixels  and  every  seventh  line,  but  the  profiles  are  drawn  at 
every  third  correlation  point  and  are  therefore  separated 
by  twenty-one  pixels.  This  keeps  the  profile  lines  from 
interfering  with  the  3-D  perception  of  the  terrain  surface. 
The  only  available  option  not  used  in  this  compilation  was 
that  of  match-point  adjustment  of  bad  points  by  interpola¬ 
tion  between  the  good  points. 

The  match-point  adjustment  option  was  not  used,  for  the 
sake  of  consistency,  because  the  results  shown  in  Figure  6 
were  used  as  a  "standard"  against  which  the  results  of 
other  tests  were  compared.  The  "other"  tests  were  made  to 
show  how  errors  can  occur  when,  for  example,  the  window 
shaping  option  is  not  used.  Match-point  adjustment  could 
not  be  used  in  the  other  tests  because  this  option  is  so 
successful  that  it  tends  to  make  bad  results  look  good  and 
therefore  leads  to  erroneous  conclusions. 

A  pocket  stereoscope  can  be  used  to  view  the  stereogram  and 
to  judge  how  well  the  profiles  conform  to  the  terrain  sur¬ 
face.  Altogether,  3265  match  points  were  computed  and  only 
28  points  were  judged  to  be  bad--a  success  rate  of  99.1 
percent.  Of  the  3265  points,  3180  points  (97.4  percent) 
were  correlated  successfully  on  the  first  try,  32  required 
two  tries,  16  needed  three  tries  and  37  needed  four.  Only 
9  out  of  37  passed  on  the  fourth  try  leaving  28  bad  points. 
Only  159  of  the  points  required  corrections  of  three  pixels 
or  more.  The  larger  corrections  indicate  instances  where 
the  correlation  process  was  "lost"  and  had  to  "search"  for 
correlation  rather  than  simply  refine  the  predicted  point. 

Since  97.4  percent  of  the  points  in  the  standard  case 
passed  on  the  first  try,  one  of  the  "other"  tests  involved 
the  case  where  only  one  iteration  was  allowed  in  order  to 
judge  the  significance  of  the  extra  iterations.  With  only 
one  iteration,  250  points  (vs.  28  previously)  were  judged  to 
be  bad.  This  translates  to  a  success  rate  of  92.3  percent 
compared  to  99.1  percent  previously.  The  92.3  percent  rate 
is  quite  high,  but  it  is  misleading  in  this  case  because  the 
operator  had  to  interrupt  the  automatic  process  15  times  in 
order  to  put  the  dot  back  "on  the  ground?'  This  kept  the 


success  rate  high. 


To  demonstrate  the  success  of  the  match-point  adjustment 
option,  the  one-iteration  test  was  run  a  second  time  but, 
when  bad  points  were  encountered,  the  match-point  coordi¬ 
nates  were  replaced  with  values  interpolated  between  good 
points  on  either  side  of  the  bad  ones.  These  adjusted  val¬ 
ues  were  used  in  subsequent  lines  to  predict  the  "next" 
match  points  and  to  compute  window  shaping  parameters.  The 
results  of  this  test  were  that  65  points  were  found  to  be 
bad  compared  to  250  when  interpolation  was  not  used.  Also, 
while  the  automatic  process  stopped  15  times  before,  now  it 
did  not  stop  at  all.  With  the  match-point  adjustment  op¬ 
tion  the  results  obtained  with  one  iteration  were  almost 
as  good  as  those  obtained  in  the  standard  case. 

DISCUSSION 


General . 

In  the  process  of  building  up  the  Digital  Interactive 
Mapping  Program,  various  options  were  tried,  tested  and 
evaluated.  Some  were  discarded  and  some  were  retained  but 
it  was  found  time  and  again  that  some  combinations  of  op¬ 
tions  worked  as  well  as  others.  The  point  is  that  there 
are  several  ways  in  which  a  suitable  algorithm  can  be  im¬ 
plemented  and  the  DIMP  is  just  one  of  several.  The  DIMP 
serves  the  purpose  for  which  it  was  developed  and  it  is  an 
excellent  tool  for  conducting  tests  for  the  purpose  of 
algorithm  development,  simulation  of  other  compilation 
schemes,  and  trade-off  analyses.  The  main  attribute  of  the 
DIMP  is  that  it  is  interactive;  and  therefore,  the  operator 
can  anticipate  when  and  where  interaction  is  necessary  and 
prevent  the  propagation  of  errors.  This  is  made  possible 
because  the  DIMP  provides  a  global  view  of  the  stereoscopic 
model  and  of  the  correlation  data  in  the  form  of  3-D  pro¬ 
file  lines.  Another  important  feature  is  that  the  operator 
can  choose  small  discrete  areas  for  compilation  and  test 
the  effects  of  specific  terrain  features  on  the  correlation 
process.  This  was  important  in  the  early  stages  of  the 
DIMP  development  when  appropriate  logic  had  to  be  designed 
to  handle  the  unique  correlation  problems  imposed  by  fea¬ 
tures  such  as  orchards.  Test  data  derived  from  orchard 
areas  eventually  lead  to  a  logic  scheme  that  worked  well  in 
both  the  orchard  and  in  general  areas  as  well. 

As  far  as  the  algorithm  is  concerned,  two  features  can  be 
signaled  out  as  having  major  significance.  First,  the 
iterative  capability  was  found  to  be  a  powerful  means  for 
maintaining  good  correlation  results  in  adverse  correlation 
areas.  This  technique  essentially  changes  the  DIMP  philos¬ 
ophy,  when  required,  from  one  of  refinement  to  a  predicted 
match  point  to  one  of  a  limited  search  operation  over  a 
small  area.  It  was  found  generally  that  90  percent  of  the 
time  a  refinement  was  sufficient  but  that  the  search  opera¬ 
tion  kept  the  process  on  track  in  the  poor  areas  thereby 
allowing  the  refinement  process  to  remain  appropriate  in 
the  good  areas.  In  other  words,  without  the  search  logic, 
the  process  would  get  lost  in  the  bad  areas,  continue  lost 
in  the  good  areas  and  thereby  negate  the  use  of  a  refine¬ 
ment  technique.  A  second  major  feature  of  the  DIMP 
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algorithm  is  the  option  to  replace  the  match-point  coordi¬ 
nates  of  bad  points  with  those  derived  by  linear  interpo¬ 
lation  between  good  points.  This  is  done  at  the  completion 
of  a  row  of  correlations  and  therefore  the  advantages  are 
available  immediately  to  correct  bad  data  in  that  row  and 
enhance  the  chances  for  suitable  correlations  in  the  next 
row  via  more  accurate  match  point  prediction  and  window 
shaping . 
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